Trace Driven Analytic Modeling for Evaluating Schedulers for Clusters of Computers

Large enterprises use clusters of computers with varying computing power to process workloads that are heterogeneous in terms of the type of jobs and the nature of their arrival processes. The scheduling of jobs from a workload has a significant impact on their execution times. This report presents a trace-driven analytic model (TDAM) method that can be used to assess the impact of different schedulers on job execution times. The TDAM approach uses an implementation of the scheduler to schedule jobs that are fed into analytic models of the computers in the cluster. These analytic models use closed queuing network methods to estimate congestion at the various nodes of the cluster. The report demonstrates the usefulness of the TDAM method by showing how four different types of schedulers affect the execution times of jobs derived from well-known benchmarks. The report also demonstrates how the method can be applied to heterogeneous computer clusters such as the ones used to run MapReduce jobs.